Method and apparatus for integrated tracking of visitors

ABSTRACT

System and method for tracking a mobile device includes: receiving unique identifications for a mobile device; filtering out the unique identifications to obtain a true identifications for the mobile device; identifying cameras relevant to movement of the mobile device; receiving video streams; generating data structures for the video streams and tracking information of the mobile device, the data structure including time stamped videos, and viewpoints of the identified cameras; utilizing the data structures to retrieve, video and tracking information for the mobile device and the user, as the mobile moves in the site; and applying analytics to the retrieved video and tracking information to analyze behavior of the user and to predict what the user will do while on site.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Patent Application claims the benefits of U.S. Provisional PatentApplication Ser. No. 62/366,276, filed on Jul. 25, 2016 and entitled“Method And Apparatus For Reversing MAC Privacy Settings; andProvisional Patent Application Ser. No. 62/376,064, filed on Aug. 17,2016 and entitled “Method And Apparatus For Integrated Tracking OfVisitors;” the entire contents of which are hereby expresslyincorporated by reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

The disclosed invention relates to electronically determining visitortraffic and more specifically to a system and method for integratedtracking of visitors to an area and predicting their future behavior.

BACKGROUND

Physical security monitoring and forensics has become a significantconcern in the past two decades. With the heightened need for securityservices in public and private environments, there has been anexponential growth of video information that needs to be reviewed,analyzed and filtered to maintain basic security monitoring. Theaccelerated pace of security video infrastructure is also placingdemands on how to monitor and analyze all the information that is beingstreamed and stored. Video information is a key component in securityand law enforcement to identify and track criminal and suspiciousbehavior. In most instances, the amount of video footage available at atypical site far exceeds the ability of the manpower needed to review itat the time of occurrence of an event.

In recent cases where the locations are very large and geographicallyspread out, law enforcement is faced with the considerable task ofcollecting all video information from all cameras in an area andmanually reviewing every frame first to identify the target individuals,then review all videos from earlier time periods to reconstruct theirpath, and finally stitch together the video stream of an event toreconstruct the entire event. This entire process only captures theimmediate aftermath of a given event, providing no clue for interceptingindividuals in real time or predicting where they will appear next. Italso does not allow for searching archival footage for past visits wherethe individual in question could be further identified, potentialconfederates/associates observed, and activities that shed light onpreparation work discovered.

Mobile/wireless devices are now commonplace among consumers and are usedregularly to search for information, content and data on products andservices across all industries. Retail and other enterprise environmentscan now use information gathered from mobile devices on their premisesto engage visitors in more meaningful ways. Providing an entry point formobile marketing, mobile devices are important to broadcast offers,information and location services to users who are on site or closethereto. However this information has low value if there is noadditional information provided about what users are doing on thepremise, where they are visiting, what their demographic information is,and pinpointing their needs and interest in a more targeted fashion.Providing an immersive environment where visitors, behavior can bepredicted and influenced is a key objective to future retail, restaurantand other service industries.

Users and visitors to retail and other public environments such astransit hubs, hospitals, schools and city streets may be immersed inzones where different types of radio based connectivity andcommunications points are available. WiFi, Bluetooth, Bluetooth LowEnergy, 3G, 4G, LTE and others can provide venues for different means ofcommunication with data and objects around the user. These technologiescan provide different forms of information and access to internet databased on smartphone apps, browser based web applications, as well asanalysis from visitor metrics collected to identify interests and needs.The providers of these technologies may be carriers, merchants, otherenterprises and cities.

These providers can use these technologies to allow local users (bothactively or passively) to connect and receive specific information andservices in the form of commercial offers, location based services andbandwidth/communications access. The technology providers can collectthe user metrics (based on permissions, for example, from theirprofiles) including social WiFi login details, location and individualdemographics for various commercial and customer assistanceopportunities. Smartphones and wearable devices (“mobile devices”)broadcast and communicate with this infrastructure providing locationinformation and IDs to the network devices located on the premisesaround them.

Along with this ID and location information, social media such asTwitter™ and Facebook™ can also be used to provide clues to problemevents about to happen or currently taking place at large venues. Inlight of the above excessive manpower requirements and error-pronenessin current physical security monitoring, there is a need for a methodand system that can automate real-time monitoring and facilitatepost-event forensics by integrating, analyzing, filtering and performinganalytics on all of these information that includes streams video, IDswith location, and information from social media.

SUMMARY

The disclosed invention manages integrated coordinate trackinginformation, video information, and social media information andincorporates a query language to facilitate the retrieval of objects(information) from various perspectives and declares rules and theirassociated triggers to perform specific processing when some an event istriggered.

In some embodiments, the disclosed invention is a method ornon-transitory computer storage medium including a plurality ofinstructions for tracking a mobile device in a site. The methodincludes: receiving, in real time, a plurality of unique identificationsfor a mobile device vising the site; filtering out the plurality ofunique identifications for the mobile device to obtain a trueidentifications for the mobile device; in real time, identifying camerasrelevant to movement of the mobile device responsive to the trueidentifications for the mobile device; receiving video streams of themovement of the mobile device from the identified cameras, time stampingthe received video streams and storing the time stamped video streams ina computer storage medium; generating data structures for the videostreams and tracking information of the mobile device, the datastructure including time stamped videos, and viewpoints of theidentified cameras; utilizing the data structures to retrieve, in realtime, video and tracking information for the mobile device and the user,as the mobile moves in the site; and applying analytics to the retrievedvideo and tracking information to analyze behavior of the user and topredict what the user will do while on site.

In some embodiments, the disclosed invention is a system for tracking amobile device in a site. The system includes: a plurality of camerascoupled to a computer network; a wireless access point for providingaccess to the computer network; a computer storage medium for storinginformation; and a server coupled to the computer network for:receiving, in real time, a plurality of unique identifications for amobile device vising the site, from the wireless access point via thecomputer network; filtering out the plurality of unique identificationsfor the mobile device to obtain a true identifications for the mobiledevice; in real time, identifying relevant cameras from the plurality ofcamera relevant to movement of the mobile device responsive to the trueidentifications for the mobile device; receiving video streams of themovement of the mobile device from the identified relevant cameras, timestamping the received video streams and storing the time stamped videostreams in the computer storage medium; generating data structures forthe video streams and tracking information of the mobile device, thedata structure including time stamped videos, and viewpoints of theidentified cameras; utilizing the data structures to retrieve, in realtime, video and tracking information for the mobile device and the user,as the mobile moves in the site; and applying analytics to the retrievedvideo and tracking information to analyze behavior of the user and topredict what the user will do while on site.

The data structures may further include a user of the mobile device,regions of interests of the user of the mobile device, a list of one ormore tracking events, video events, social events, area events, webrequest events, partner events, trackables, cameras, social threads, andareas, and wherein each list includes identifications that referencesaid each list. The prediction of what the user will do while on siteincludes one or more of predicting a future travel path of the mobiledevice, predicting what web sites the user will visit, and what productsor services the user is likely to purchase and may be used for providingproducts, services and offers, product placement and management,staffing support and scheduling, security or law enforcement services,and/or object or person tracking.

In some embodiments, the disclosed invention further identifies previouspaths of the user, matches the previous paths with the video streams ofthe movement of the mobile device for further identification of theuser, and generates alert information when the user returns to the site.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood with regard to the followingdescription, appended claims, and accompanying drawings.

FIG. 1 Illustrates an exemplary environment, according to someembodiments of the disclosed invention.

FIG. 2 depicts a RealStory object and its content, according to someembodiments of the disclosed invention.

FIG. 3 illustrates exemplary subsystems of a server, according to someembodiments of the disclosed invention.

FIG. 4 shows a server with social media streams integrated, according tosome embodiments of the disclosed invention.

FIG. 5 depicts a predicted object and its probabilistic content,according to some embodiments of the disclosed invention.

FIG. 6 shows an exemplary simplified process flow, according to someembodiments of the disclosed invention.

FIG. 7 is an exemplary display screen depicting images from alllocations where the targeted devices were seen, according to someembodiments of the disclosed invention.

FIG. 8 is an exemplary display screen displaying timelines from thevideo footage from the cameras, according to some embodiments of thedisclosed invention.

FIG. 9 is an exemplary display screen showing a view for the past visitsand the location of those visits, according to some embodiments of thedisclosed invention.

FIG. 10 is an exemplary simplified process flow for predicting locationand direction of travel, according to some embodiments of the disclosedinvention.

DETAILED DESCRIPTION

In some embodiments, the disclosed invention augments and improvesphysical security monitoring technology and the storage of monitoringinformation, which may include identifications coupled with locations,video, and social media in order to beneficially realize automatedreal-time triggers and notifications, as well as improve forensicanalysis technology. This beneficially realizes much less error.

In some embodiments, the device and method of disclosed inventionfilters out the unwanted observations from a measurement system that isseeking to identified the mobile devices on a location and measure thetraffic and its nature for the site. The invention identifies real(same) devices and observations from the masking random devices andobservations, and filters the masking random devices and observationsfrom the results.

In some embodiments, the device and method of disclosed inventionelectronically detects and measures visitor traffic for a site, appliescorrection filtering to the measurement data for higher accuracy, andanalyzes the filtered data to apply analytics to analyze behavior of theusers/visitors and optionally to predict what users will do while onsite. The technology may be used in retail, public and governmentplaces, service organizations, entertainment industries/venues, securityand law enforcement applications and the like.

In some embodiments, the device and method of disclosed invention linklocation and user information from mobile devices (includingsmartphones, mobile computers, and wearable clothing and other wearabledevices) that provide their IDs by wireless communications, with videostorage devices to capture the time stamp details from a video stream,to provide rapid analysis and real time predictive/alert information aswell as forensic details for later analysis. Using analysis ofinformation captured from mobile users allows insights into visitorpatterns and can be used for forensic and context analysis of the event.Further, all the video from the time of the event can be packaged forelectronic delivery to law enforcement, security, insurance and productpromotion organizations for further analysis and prosecution.

FIG. 1 Illustrates an exemplary environment, according to someembodiments of the disclosed invention. As shown, a server 101 isdeployed as a listener to streams of information in a wireless areanetwork 102 system. The streams of information arrive as tracking data107 and video (or image) data 108. The Tracking data 107 is generated byany wireless device such as smartphone 103, simple cellphone 104, tablet105, laptop 106, smart watch 109, or any other similar device. Multiplevideo streams are also generated, one video stream per camera 110, whichis deployed in a specific locale. The server 101 aggregates thesestreams to produce many objects that represent an organized structure oftracking and video clip segments in various meaningful ways to representa “story” or scenario.

The server 101 performs the above-mentioned stream aggregation ofinformation together as a means to provide a complete detail of thescenario on various specific objects of interest. Examples of objects ofinterest include: a person, a group of people, a camera's viewpoint, andspecific areas or regions of interest. In some embodiments, the servercombines this information with time stamped video information recordedby cameras 110 and other video/audio capture devices, if any. Byaccessing the recorded video information from a database connected tothe Internet or video recording storage device, the system canfacilitate quick analysis and filtering of video streams of individualsinvolved in a targeted event and display them on a single screen to showboth the locations of individuals on a map and the associated videostreams from cameras in the path of the individuals to view theirprogress. In some embodiments, the invention depicts the predictedbehavior (e.g., a predicted future path of the individuals) on a map ona display screen.

In some embodiments, the electronic measurement system and method of thedisclosed invention allow enterprises, merchants and service providersgreater control over the information from visitors and what the visitorsare doing when using their mobile devices on site. In some embodiments,information is captured on visitors' traffic, duration of their stay onthe site, physical and web locations visited by them, and details fromsocial WiFi login, apps used, and web browsing actions. In someembodiments, the information is captured in real time for immediateapplication to analytics, services and/or products. Pooling thisinformation from different sources (point of sales terminals, iBeacondevices and/or WiFi access points) enables the invention to analyze thebehavior of the visitors and predict what the visitors will do, based ondifferent metrics and prior visits to the site or other sites. Thisinformation can be used to provide services and offers but can also beused by the location staff for different goals such as inventory/productplacement and management, staffing support and scheduling, security/lawenforcement services, object or person tracking on site (child,equipment, cleaning crew, VIP) and site layout and therefore improvesinventory management, labor and employee management technologies,security and identification technologies, and lost-and-found technology,for example.

In some embodiments, the information is collected via log analysis ofthird party devices as well as inline traffic inspection software anddevices. For example, by looking at URLs, DNS and all destinations todetermine interests and looking at searches performed by theindividuals, products viewed and types of locations visited, aconsolidated view of a user's online mobile browsing behavior (whatwebsites to visit or visited) can be collected for post processing.Then, what product and service the user is likely to purchase can bepredicted from the user's behavior pattern.

The analytics information also provides details on real time trafficmovements that can be displayed, for example, as heat maps. Heat maps(real-time, historical, dwell and frequency of visit metrics) todetermine locations of interest are generated for the merchant or thelaw enforcement. In some embodiments, the invention animates heat mapsto show real time tracking, visitor density location popularity and flowanalysis of the visitors within the site's infrastructure range.Animated heat maps provide a directional vector to support locationfinding services that can be zoomed in to unique individuals orequipment and to identify traffic directional movements.

In some embodiments, the information derived and assembled by thedisclosed invention includes presence analytics, such as total visitortraffic, dwell time duration, real time visit capture rate and the like;loyalty analytics, such as repeat visitors, visit frequency andduration, how recent were the visits, and the like; engagementanalytics, such as conversion and bounce rates, social WiFidemographics, visitor contact information, and the like; and WiFi usageanalytics, such as websites visited, page views, product/price views,and the like.

The invention provides quick matching and recognition of device IDs andthe time stamps that the device was seen by a location's wirelessinfrastructure with the time and location information to rapidly accessthe associated video information captured by security and other digitalvideo recording equipment from the same location as the wirelessinfrastructure devices. Matching this information can be done in realtime, to quickly identify the target ID to track, access the cameraobservation information from other locations where the wirelessinfrastructure viewed the target individual or individuals and, usingadvance path analysis, predict the paths where the individuals mayappear next in order to focus efforts on those local security cameras.

The same information may be used to look back in time to identify theoriginal and previous paths of the targeted individual, providenotification of origination details, for example, where the personparked their car upon arrival, and identify any accomplices. Review ofthe archive of past visit history (e.g., from a database) can be quicklymatched with the video streams from those visits in real time forfurther identification of the user and potential past confederates.Alert information of future visits, can also be provided for future usein case a targeted individual or confederates return. Alert informationmay include a sound alert signal, a visual alert signal, photos of thetargeted individual or confederates, any information about targetedindividual or confederates, and/or their past behaviors.

Information garnered from visitors can also be used to trigger ortransmit messages from interactive terminals and digital signage screensthat can be tailored to the user based on the goals of the site, toprovide an immersive experience that is customized to the visitor. Forexample, using specific data from the analytics processes and userspecific information captured and cataloged from WiFi sensors thatidentify active WiFi signals from devices. Examples of WiFi sensorsinclude Bluetooth sensors and RFID sensors that sense active devices intheir respective frequency ranges, access points, social login detailsand/or cameras allows specific triggers to be generated. The triggersare designed to be sent to media and content servers in order to changethe content presented on the digital signage to the specific user. Bylocating the user near a specific digital sign and using the customerspecific information of their location and other identified details(from social media, other analytics information from this and past sitevisits), the invention allows targeted content to be presented on thedigital sign. Current digital signage systems have a media server and/orcontent delivery network to program content to be displayed at alocation in a preprogrammed fashion. Triggers generated from knowledgeof the customer through analytics can request from the system that avery specific message be displayed in real time to the local user.

The information can also alert merchants to user behaviors that affecttheir sales and provide them with different means to communicate offersor improved branding and loyalty opportunities for their visitors. Inother cases, visitors may use their internet access within stores, tocompare prices and shop online without purchasing from the brick andmortar store (“showrooming”).

When searches are conducted on premises, the merchant or vendor is at adisadvantage to online product vendors because they cannot control whatthe consumer is doing online in their store. By utilizing a captivesocial WiFi offering according to some embodiments of the invention, themerchant can provide access to the internet to users but can now controlthe access provided and receive information on product and price datathat users are searching, and provide competitive offers or services asalternatives before the user leaves or purchases from the online vendor.

Typical consumer behavior while on a (retail) site is to search forproducts from competitors, pricing, reviews, coupon offers andcompetitive alternatives. The merchant/vendor can use the technology ofthe disclosed invention to intercept and monitor the searches andproduct/price views through the technology's capture and displayfunction. In some embodiments, the invention provides additionalfeatures to enable the merchant to send an alternative price, product,competitive service or solution, to the consumer. In some embodiments,the invention features additional capabilities to filter, preset andsend various offers, ads to the mobile device, and combined with thedigital signage feature, to display relevant information to engage theconsumer while they are still on location.

FIG. 2 depicts a RealStory object structure and its content, accordingto some embodiments of the disclosed invention. As shown, a “RealStory”object 201 comprises a list of TrackingEvents 202, list of VideoEvents203, list of SocialEvents 204, list of AreaEvents 205, list ofWebRequestEvents 206, list of PartnerEvents 207, list of Trackables 208,list of VideoCameras 209, list of SocialThreads 210, and list of Areas211. Each of the lists include IDs that reference the objects uniquely.Note that the first four Event Types 214 reference associated ObjectTypes 215 such that all associated Object Types 215 can be retrievedfrom the Event Types 214. In order to perform the opposite, one mayretrieve the associated Event Type 214 objects for a given Object Type215 by submitting a JOIN query with the Object Type's ID and the ObjectType ID field in the Event Type 214. For example, Object Type IDs areTrackableID, VideoCameraID, SocialThreadID, etc. which are maintained inthe Event Type 214 structures.

Some embodiments of the disclosed invention uses these unique IDs torefer to objects in a database wherein the ID may be employed to look upa row in a relational table in order to retrieve the objectcorresponding to the ID for a specific Object Type 215. In suchembodiments, each table corresponds to a specific Object Type 215, forexample, TrackingEvent 202, VideoEvent 203, SocialEvent 204, etc. Also,Trackable 208, Camera 209, etc., would each represent an Object Type (orclass in an object oriented environment) and are included in a table foreach Object Type 215. Such tables may be implemented in any type ofdatabase.

Some embodiments of the disclosed invention employ TrackingEvent 202which are comprised of a TrackableID that corresponds to a person orobject that is trackable. While a Trackable 208 is most often a personwith a wireless device, it might be an inanimate object with a wirelessdevice that can be tracked by the server 101. Typical equipment that isWiFi enabled includes medical equipment used in hospitals that use WiFito connect to and update an EHR database for a patient, in commercialsettings, devices such as printers, portable sales terminals (such as atablet with an attached credit card reader), inventory scanners, andhand held data input devices used in factories and retail stores toupload information to a cloud based database.

In some embodiments, a Trackable 208 may have multiple associated mediaaccess control (MAC) addresses. This is because it actually has multiplesuch addresses, but it could also be because it was given a new wirelessdevice associated with it to replace an earlier one. The inventionrecords and stores this relationship to maintain for Trackables 208.

In some embodiments, a TrackingEvent 202 identifies the primaryTrackable 208 and includes a specific MAC address identifying aTrackable 208 device along with a history of locations listed with(timestamp, x, y, z). The Trackable 208 may also maintains a List ofTrackingEventIDs since each TrackingEvent 202 stores a Mac address withthe (x, y, z) coordinate path it traversed over a time period.

In some embodiments, the disclosed invention maintains a Trackable 208that denotes the type of the Trackable 208 and a name which is uniqueacross all Trackables. A Trackable may also be generically“StationaryObject” but more specifically “Label” or “PriceTag”. Theinvention may employ any collection of descriptive Object Types 215.

Some embodiments of the disclosed invention include a list of sixassociated information facts comprising (Username, Lastname, Firstname,Site, MAC address, Timestamp) in the Trackable structure 208. Thisprovides the very important association of a username andLastname+Firstname to a MAC address, which ultimately provides knowledgeof precisely who is present and moving around. Once skilled in the artwould recognize that the disclosed invention is not limited to a list ofsix associated information facts, rather, other numbers of associatedinformation facts are possible and within the scope of the presentinvention.

In some embodiments, the disclosed invention maintains a VideoEvent 203that refers to a specific VideoCameraID along with the start and endtime of the video. The camera referenced in the VideoEvent 203 may beemployed to look up the actual video clip from a Video Object 209,wherein the video clip is extracted from the stored video based on thestart and end time. The Video Object 209 may also have a unique name orID. The Video Object 209 being associated with a specific camera, alsocontains a list of AreaIDs 211 that corresponds to the exact areas, andhence, (x, y) locations for which the Camera sees the object(s). Thus,these camera areas are used to know when a TrackingEvent 214 traverseswhere a camera has sight. This may be achieved when each Video frame isstored with its timestamp or the time that a frame was taken can becomputed based on an assumption that each frame was recorded at aregular interval, for example 30 frames per second.

Some embodiments of the disclosed invention retrieve the video clipusing a video management system (VMS). In this case, the video clip isretrieved with a start and end time. The VMS provides retrieval of avideo clips based on VideoCameraID and start and end time.

Some embodiments of the disclosed invention incorporate a SocialEvent204 that includes a SocialThreadID 210 and start and end timestamp. ASocialThread Object 210 comprises a SocialMediaType that corresponds toone of the well-known social sites where posts and various interests areshared, such as Facebook™, Twitter™, YouTube™, or others. It alsoincludes a Topic attribute that gets computed based on analysis of postsin the thread. The posts may be stored as a list of (timestamp, postUrl)pairs, one for each post or submission to such a social site. Thisprovides the means for a SocialEvent 204 to be defined that has a moreprecise time period than the entirety of a SocialThread 210 that maycomprise less important items.

Some embodiments of the disclosed invention incorporate an AreaEvent 205which includes an AreaID, the MAC address that was present in the area,and the start and end timestamps for which the tracking information waspresent in the AreaEvent 205. The AreaEvent 205 facilitates asummarization of where the RealStory 201 in question occurred and atwhat date and time. This is also useful for, not only an overview ofwhere somebody traversed, but also for triggering some rules. The AreaIDrefers to an Area object 211 stored in the Area table. Area objects 211includes a name, acting as a label, and a List of (x, y) coordinatesthat represent a closed region.

Some embodiments of the disclosed invention incorporate an Area object211 that has a Boolean “CameraViewport” denoting whether or not the arearepresents a camera viewport. Some embodiments of the disclosedinvention manage a WebRequestEvent 206 that represents a single HTTPrequest. It maintains a WebsiteID that is associated with a Websiteobject 212. Having this Website object 212 association provides easyquery and analysis per Website across all WebRequestEvents 206.WebRequestEvent 206 may also include a start-timestamp for when an HTTPrequest was made along with the request URL, request header, and requestbody, and an end-timestamp for when the responses are received. AWebRequestEvent 206 may also include the response code, a header and abody. The WebRequestEvent 206 may also include a computed“InterestingData” field which stores a dictionary of interesting requestand response parameters and their values. This simplifies obtaining thedata of interest from WebRequests. The “InterestingData,” includinginformation for username, site, last name, and first name, is copied tothe corresponding Trackable object 208 as well.

Some embodiments of the disclosed invention incorporate a PartnerEvent207 when another MAC address has been determined to correlate (traveltogether) with one of the TrackingEvents 202 per includedTrackingEventID. The PartnerEvent 207 also includes a start and an endtimestamp to denote the portion of the TrackingEvent 202 that thePartner was correlated with the TrackingEvent 202. In such embodiments,it is possible that there may be more than one PartnerEvent 207 whichmay represents a group of more than two people found to have traveledtogether.

FIG. 3 illustrates exemplary subsystems of a RealStory server, accordingto some embodiments of the disclosed invention. As shown, a server 330continuously listens to a wireless access point 332 for the incomingtracking information 337 and video data streams 338. A tracking streamssubsystem 301 and a video streams subsystem 302 preserve the order ofthe incoming tracking and video data in the order it arrives so that theincoming events can be examined for further processing by a customstreams subsystem 320 or a rule manager 306. The custom streamssubsystem 320 provides the means to deploy custom processing totransform the raw tracking stream 301 and raw video stream 302 intoother kind of meaningful data streams. All of these data streams aredirected (312, 313, and 314) to a rule manager 306 which in turn, emitsevents 318 and/or additional streams 319 of events.

The rule manager 306 is able to recognize specific events or patterns ofevents, which may even be restricted to occur in a time window, in orderto emit further events 318 and streams 319 of events. The rule manager306 may also employ state read 317 from a RealStory manager 305. TheRealStory manager 305 references and retrieves stored video clips asnecessary to store and maintain RealStory object structures. A trackingmanager 303 stores the incoming data from the tracking streams 301.Similarly, a video manager 304 stores the incoming video streams 309. Insome embodiments, all subsystems log occurrences that take place tomaintain an audit via a History Manager 307. The RealStory manager 305also maintains the ability to retrieve (315, 316) the necessary trackingand video objects from their respective managers (303 and 304), asnecessary.

FIG. 4 shows a RealStory server with social media streams integrated,according to some embodiments of the disclosed invention. As shown, aserver 330 is integrated with a variety of social media 403. In thiscase, the social media 403 provides SoicalMediaEvents 404 in a Socialmedia stream 401 to a RealStory manager 305 for reception by a socialstreams subsystem 401. This subsystem then stores each

SocialEvent to the social manager 402. The Objects stored in theRealStory manager 305 may reference social media. Additionally, socialstreams 401 comprising SocialMediaEvents are streamed to a customstreams subsystem 407 as well as the rule manager 306 for ruleprocessing. The rest of the subsystems depicted in FIG. 4 operatesimilar to those illustrated in FIG. 3.

Some embodiments of the disclosed invention include various adaptors tointerface to various social media. This provides a unified architecturefor multiple social media and sharing services implementations.

FIG. 5 depicts a predicted object and its probabilistic content,according to some embodiments of the disclosed invention. In theseembodiments, while the object may be similar to the object 201 of FIG.2, its core content emphasizes a PossibleStory object 502 comprising aprobability, a start-timestamp, and an end-timestamp, as well as asubset of a RealStory object containing TrackingEvents, AreaEvents, andPartnerEvents. Each PossibleStory object 502 also comprises a Summaryobject 503. The probability is a percentage from 0 to 99.9999%expressing the probability of the PossibleStory object 502 to take placein the future over the period start-timestamp to end-timestamp. Alltimestamps on reachable object children from a PossibleStory object 502should be limited to be within the PossibleStory object 502 start andend-timestamps. A PredictedStory object 501 references multiplePossibleStory objects 502 and the probabilities across PossibleStoryobjects 502 for a specific time period should add up to 100%.

In some embodiments of the disclosed invention, the Summary object 503also includes means and variances of past behavior associated with EventType objects. For example, an embodiment employing a Summary object 503provides visitation frequency mean and variance wherein the summaryobject 503 would be associated with an AreaEvent, which represents thevisitation destination. This summarized past behavior is useful forprediction under the assumption that past behavior generally repeatsitself. Thus, since a Summary object 503 is associated with aPossibleStory object's probability, any regular past behavior may beassigned 99.9999% probability. As another example, two differentbehaviors may be characterized in two distinct PossibleStory objects502, each with a probability percentage based on the percentage of timesthey each occurred.

Some embodiments of the disclosed invention, perform sophisticatedanalysis on TrackingEvent paths and incorporate a Summary object 503associated with predicted TrackingEvent paths. Any number of methods maybe employed to generate the predicted path. For example, an embodimentmay compute an average of similar multiple paths across multiple timeperiods found and assigns small probability to paths that are taken onceor a few times.

In some embodiments of the disclosed invention, the Summary object 503may further include means and variances of past grouping behavior withothers such that the Summary object 503 would be associated with aPartnerEvent.

Some embodiments of the disclosed invention include a“GeneratedDescription” attribute which is filled in with arbitrary textgenerated by the embodiment that may include not only mean and varianceof frequencies but also any other past behavior generalizations or evennoteworthy occurrences. Such embodiments may incorporate a simplenotation in such text to declare data that may be analyzed. For example,something as simple as ({fieldname}: {fieldvalue}) may be used.

FIG. 6 shows an exemplary simplified process flow, according to someembodiments of the disclosed invention. As shown in block 602, a mobiledevice reveals its unique IDs (e.g., (MAC ID, Bluetooth ID, SIM ID,etc.) to access a WiFi access point and/or other network or sensornodes, in a venue/location. Optionally, the unique IDs for the mobiledevice are filtered (as explained above) to obtain a true unique ID,which does not include false or repetitive IDs. For example, somefiltering methods may be used to filter out any ID randomization of themobile devices to obtain more accurate information. For instance, via adisplayed dashboard, the system may be queried for all the device IDs inthe location and filtered by time, in real time. The access point(optionally, with its sensor channel) collects coordinate and MAC IDinformation from the connected wireless device, in block 604. A typicalWiFi communication operates in the 2.4 and 5 Ghz spectrum. In eachspectrum, there are multiple channels available to be used by devices toconnect, parsing out the connections across multiple channels helpsavoid signal interference when many devices are using the same spectrum.In the case of 2.4 Ghz, there are 11 to 14 channels depending on theregion, for the 5 Ghz spectrum there are about 23 separate channelsavailable. This makes a higher density of connections possible).

In block 606, the invention identifies the cameras (and optionallysensors) relevant to, for example, in the vicinity and/or view point ofthe mobile device or an event), collects information from sensors,cameras and other relevant nodes and send the information to a analyticsmeasurement device located on the server. Each camera has an areaassociated with it that corresponds to the coordinates of the localethat the camera can see. The disclosed invention then knows which camerais of interest (relevant) for a wireless device when the devicetraverses a specific coordinate or coordinates.

The server collects the information from all access points to know whichdevices are traversing which coordinates and also which video clips areassociated with each device, based on the area data associated with acamera. Optionally, the analytics measurement device searches a databasefor the device IDs for the previous locations seen and their timestamps, in block 608.

In block 610, the obtained information is analyzed for real time mappingof the location and predicted future path of the mobile device. Theinvention also accesses video storage device(s) and requests videosegments based on the time stamp and camera locations, in block 612. Inblock 614, the invention organizes and displays the video streamsegments in order of the time stamps and camera locations for eachdevice ID that is being tracked. For example, the video footage from allknown cameras in the identified locations and time frames are displayed,as s in FIG. 7, which is an exemplary display screen depicting imagesfrom all locations where the targeted devices were seen, according tosome embodiments of the disclosed invention. The video footage from eachphase of the event may be consolidated into a single video thread, asshown in FIG. 8, which is an exemplary display screen displayingtimelines from the video footage from the cameras, according to someembodiments of the disclosed invention. In some embodiments, the deviceand method of the invention perform an initial review of the device IDsand information about each device to eliminate, for example, employees,equipment and non-relevant IDs. For example, by using the travel andbehavior patterns, the invention is capable to distinguishing thenon-relevant device IDs from those who are indeed visiting the location.This way, the device(s) of interest are quickly narrowed down andaccurately identified.

Previous locations and their related video images may also be optionallyidentified in block 616. The invention also optionally searches archives(e.g., a database) for previous visits of each mobile device, in block618. For instance, for each device ID, a history and path analysis isdisplayed including all previously seen devices and the travel path maybe mapped to a map of the facility, showing the locations and dwelltimes of the targeted device IDs. This information may be analyzed toidentify potential accomplices or related parties. For example, in acase of a security breach, the invention analyzes past visit locationsand the associated video frames to further a security investigation, toimprove identification of the users of the mobile devices, as shown inFIG. 9, which is an exemplary display screen showing a view for the pastvisits and the location of those visits, according to some embodimentsof the disclosed invention.

In block 620, the direction of travel is optionally predicted and may beidentified on a map, depicting a map of device movements and predictedtravel path, according to some embodiments of the disclosed invention.

The invention then analyzes the video information from the cameras thatare in the travel path to follow the mobile device being tracked. Theorganized information may then be sent (e.g., streamed) to a remotelocation, for example, a cloud, merchant, and/or security or lawenforcement personnel. In block 622, the information about the devicesare saved in a database and notifications or alerts may be transmittedif the same (suspicious) mobile device visits the same site in future.Moreover, the device information and video streams can be packaged andelectronically sent to interested parties, such as law enforcement ormerchants along with the device IDs to identify phone numbers andaddresses of suspects.

This way, for example, security or law enforcement can be alerted topossible origination of the suspected event and potential accomplices(as well as their travel paths). Additional face recognition softwarecan be applied for improved identification of individuals during theevent or post processing of the images. Alerts can be automated totrigger alarms or other actionable items and location and warningmessages may be sent to relevant people (e.g., employees or security) inthe direction of suspect's travel. The processes of the disclosedinvention may be executed by one or more process included associatedmemory and circuitry, for example, a server, an access point and/or asmart camera, using instruction stored in a non-transitory computerstorage medium.

In some embodiments, the invention uses predictive analysis to indicatethe next locations where the individual may approach in order to focusattention by security on those cameras as well. The system can look backin time on multiple device IDs to narrow down the targets and searchback in time to view videos from the beginning of an event. The systemcan review archival video information along with visitor ID tracking, toview earlier visits to help identify past behavior, accomplices andconfederates working with the individual targeted.

FIG. 10 is an exemplary simplified process flow for predicting locationand direction of travel, according to some embodiments of the disclosedinvention. In some embodiments, prior to any PredictedStory request andthe corresponding query processing process flow, however, there is acontinuous collection and computation taking place which continuouslyupdates the information necessary for accurate area traversalpredication results. In some embodiments, the continuous processing isas follows:

-   -   Compute and store the sequence of areas visited with timestamp.    -   Continuously compute and store the following Dictionaries per        MAC address and per aggregate of all MAC addresses:        -   2 Level Dictionary with time→AreasVisitedList.            -   Key: Average time from Arrival, Value: List of Area                Labels already visited        -   2 Level Dictionary Area→Average time.            -   Key: Area Label, Value: Average time visited by from                Arrival        -   3 Level Dictionary with Nth Move→[Area→Probability]            -   Key: N, [Key: Area Label, Value: Probability of going to                next] (N=0 represents outside the local environment,                wherein the probabilities represent the first move into                the environment).    -   An aggregate list of paths of areas of all MAC addresses is        continuously computed.

It is computed by reading the Nth Move→[Area→Probability]Dictionary,which is read for N=0, 1, . . . and an “Area Path” constructed for eachpossible path up to the end time in the predictive query submitted. Theprobability of each complete path is easily computable by multiplyingthe probabilities of each area for N=0, 1, . . . traversed together torealize an aggregate probability for the whole path.

The same continuous computation is performed as above for eachindividual MAC address when data is received.

Referring now to FIG. 10, as shown in block 1001, a query for thepredicting the location and/or direction of the travel for a visitorwith a mobile WiFi device is submitted to the system. When a request(e.g., PredictedStory) is received, the intermediate data, whichincludes two sets of lists of area traversal paths, is retrieved (e.g.,#3 and #4 above). In block 1002, each possible complete path of eachareas traversed and its associated probability is retrieved for theaggregate of all MAC addresses. In block 1003, each possible completepath of areas traversed and its associated probability is retrieved, forthe specific requested MAC address.

In block 1004, each complete path probability for the aggregate MACaddresses is multiplied by Alpha and in block 1005, the individual MACaddress area probabilities are multiplied by Iota. That is, in block1006, the lists retrieved in blocks 1002 and 1003 are combined bymultiplying each set of Lists by a weight factor, an aggregate weightfactor of “Alpha” and an individual weight factor of “Iota,” forexample. Different embodiments of the present invention can assigndifferent values for Alpha and Iota based on their distinct strategies.Some embodiments of the present invention assign the values for Alphaand Iota based on the number of visits to the environment. In someembodiments, for computing the final PredictedStory with Alpha and Iota,the weight factors are computed as:Alpha=1/(Number of Visits+1)Iota=(Number of Visits)/(Number of Visits+1)

This example allows Alpha to have a 50% affect after the first visit butthen weight more and more the individual tracking history the morevisits are recorded. With a heuristic for computing Alpha and Iota, thefinal PredictedStory comprising multiple Area Traversal Paths withassociated probability can be computed.

Now that the area traversal paths and probabilities have been computed,the possible tracking coordinate paths and associated probabilities arecomputed. Similar to continuously updating area traversal pathsprobabilities subsystem, prior to any PredictedStory request and thecorresponding query processing process flow, the information necessaryfor accurate tracking coordinate predication results are continuouslyupdated. In some embodiments, the continuous processing is as follows:

-   -   Compute and store the average of all tracking 3D coordinate with        timestamp information per MAC address as well as per Aggregate        of all MAC addresses. This may be accomplished by breaking up        the 3D coordinate track for a MAC address into segments where        each segment corresponds to the Area it traverses. For an        individual area, each of the coordinate paths that traversed the        area is averaged together by first assuming that the point in        time that the segment first entered the area is time zero. The        rest of a segment is assumed to include coordinates with        timestamps for the number of milliseconds since the first entry.        All of the segments that traversed a specific area are then        averaged by averaging the coordinates that correspond to the        same time since entry to the area.    -   Some segments will remain in a specific area for much longer        than other segments. In this case, when averaging in a segment        that goes longer than other segments, the segments that stopped        at a particular time will average in (0, 0, 0) beyond the time        the segment has recorded coordinates for. This presents the        likely coordinates based on the information known.    -   The resulting averaged segments are computed per area for each        MAC address and for the aggregate of all MAC addresses. The        segments can be stitched together to form a coordinate path. The        stitch sequence is similar to the area sequences assembled        earlier and the probability assigned to a stitched coordinate        path is the similar to its corresponding area path for which the        probability is already computed. In some embodiments, one more        operation is performed for the stitching—the timestamps assigned        to each area's average coordinate point of entry is the average        time found in the already computed “Area→Average Time”        Dictionary. As such, any segment that exceeds the time for the        average point of entry into the next stitched segment, must be        trimmed to the time that meets the average time for the        difference between the start of the next segment and next area        and the start of its segment and area.

This results in an area path and a tracking coordinate path, each withan associated probability so that the collection of both area paths eachwith the assigned probability and tracking coordinate paths each withassigned probability may be assembled in a single PredictedStory, as theanswer to a PredictedStory query.

Referring back to FIG. 10, as shown in blocks 1007 to 1013, theinformation for answering tracking coordinate paths and theirprobabilities are retrieved and computed. In block 1007, each possiblecompleted stitch” path of tracking coordinates and its associatedprobability is retrieved for the aggregate of all MAC addresses. Inblock 1008, each possible completed stitched path of trackingcoordinates and its associated probability is retrieved for the specificrequested MAC address.

In block 1009, each complete path probability for the aggregate MACaddresses is multiplied by Alpha and in block 1010, the individual MACaddress area probabilities are multiplied by Iota. In Block 1011, theArea→Average Time Dictionary is retrieved. In Block 1012, the stitchedtogether tracking coordinate paths have timestamps adjusted at the startof each stitched segment based on the Area→Average Time Dictionary. InBlock 1013, the list of tracking coordinate paths for aggregated Macaddresses have their probabilities multiplied by Alpha while thetracking coordinate paths for the specific Mac address have theirprobabilities multiplied by Iota after which the paths are combined intoone set of complete tracking coordinate paths with probabilities.

The information from radio-based devices may include logs of mobiledevice IDs (e.g., MAC addresses), time, location (geographic and basedon radio power strength), and login information collected when the Wi-Fi(e.g., guest access or customer portal) on the location is used by theindividuals. In some embodiments, the information is collected from oneor multiple locations and sent to cloud-based or private on-site serversfor processing to generate real time detail information about bothindividuals and groups. This information allows the invention tointerpret behavior and actions and to combine this with information fromthe video storage or live video feeds for immediate action or postprocessing for marketing, promotional and/or forensic analysis.

The analytics process using device IDs and path analysis, can be appliedin real time to predict the travel path and target the relevant cameras,in real time, for immediate tracking. Forensic review can be doneafterwards by combining video information of the event in question andpast video information from previous visits of the identified devices.This information combined with additional demographics and otherdiscernible details from a database and/or from social media improvesinvestigation of the individuals involved.

In some embodiments, the invention collects information in real time toreport the exact location and video feeds from the specific cameras nearthe individual as he moves through the location to, for example,merchants, marketing companies, site security and law enforcement and/oremergency responders. In some embodiments, alerts for future visits canbe automatically generated to continue monitoring during new visits oralert authorities to the individuals' new location. Combining the videothread of the event, with path analysis and heat maps of the visitprovides new insights for to manage a situation and to conduct furtherinvestigation. The invention alleviates the enormous burden of reviewingmany hours and many video streams to quickly search and narrow down thetarget for identification and tracking.

In some embodiments, the detail information on visitor traffic, currentlocation, predicted location and demographics is analyzed to providereports and real time charts. Moreover, this information may also beused to customize and tailor the visitor's experience, offers andservices for each visitor, individually. For example, based on thisdetail information, commercial offers, coupons, location based servicesand product finding and customer support may be customized andtransmitted to the individual visitors.

The commercial offers, coupons, location based services and productfinding and customer support may be communicated to the targeted usersin a variety of known methods, such as, email, SMS, Text, social mediaplatforms, loyalty cards from the merchant, captive social WiFi portalwith custom HTML, and the like. This way, merchants can use the analysisof visitor transit patterns and interaction with their mobile devices,to improve services, create better visitor environments or adjustinventory, and manage their staff and operations. Furthermore, theinformation may be used for theft and security trends and salesmanagement information.

In some embodiments, the disclosed invention filters out the unwantedobservations/information from a measurement system that is seeking tomeasure traffic in a location, such as a retail shop or a public place.The invention then distinguishes the real observations from the random(false) observations that mask the real (repeat) mobile devices.

Using the different data sets, an analytics engine of the disclosedinvention processes the information to present highly specific detailson the device owner in order to generate targeted content, alertrelevant parties and/or trigger other equipment (such as digitalsignage, touch tablets and kiosks) to present relevant data to thetargeted device owner.

In some embodiments, the measurement system provides observations withseveral fields, such as: observed MAC address, observed number ofdetections, first seen, last seen, signal strength, and the like.

In some embodiments, the disclosed invention retrieves an object havinga unique identifier including event objects for the same uniqueidentifier. The event objects include tracking events each including atime stamped list of (x, y, z) coordinates, area events where eachcomprises a start time stamp and an associated area with a list of (x,y) coordinates corresponding to the precise region that a camera seesand continuously records, and video events where each comprises a starttimestamp and end timestamp corresponding to the precise time periodthat a tracking event traversed a camera's precise associated area andincluding the synchronized video clip. The invention may further includean area event and an associated area with a list of (x, y) coordinateswherein the area does not correspond to a camera and partner events thatare associated with the unique identifier.

In some embodiments, the disclosed invention may further include a querylanguage that provides the means to retrieve an object that representsthe predicted behavior of the object in terms of possibility objectseach having a probability percentage assignment. Each possibility objectmay include a start and end timestamp to denote a time period, aprobability of occurrence within the time period, and one or moreevents.

Some embodiments of the disclosed invention include a facial recognitionfeature to register facial features in the custom stream so thatrecognized faces are provided by the associated custom stream and forfurther rule processing.

It will be recognized by those skilled in the art that variousmodifications may be made to the illustrated and other embodiments ofthe invention described above, without departing from the broadinventive scope thereof. It will be understood therefore that theinvention is not limited to the particular embodiments or arrangementsdisclosed, but is rather intended to cover any changes, adaptations ormodifications which are within the scope of the invention as depicted bythe appended claims and drawings.

What is claimed is:
 1. A method for tracking a mobile device in a site,the method comprising: receiving, in real time, a plurality of differentidentifications for a particular mobile device visiting the site,wherein the plurality of different identifications include false orrepetitive identifications for the particular mobile device; filteringout the plurality of different identifications for the particular mobiledevice to obtain a true identification for the particular mobile device,wherein the true identification does not include false or repetitiveidentifications for the particular mobile device; in real time,identifying cameras relevant to movement of the particular mobile deviceresponsive to the true identifications for the particular mobile device;receiving video streams of the movement of the particular mobile devicefrom the identified cameras, time stamping the received video streamsand storing the time stamped video streams in a computer storage medium;generating data structures for the video streams and trackinginformation of the particular mobile device, the data structureincluding time stamped videos, and viewpoints of the identified cameras;utilizing the data structures to retrieve, in real time, video andtracking information for the particular mobile device and the user, asthe particular mobile device moves in the site; and applying analyticsto the retrieved video and tracking information to analyze behavior ofthe user and to predict what the user will do while on site.
 2. Themethod of claim 1, wherein the data structures further include a user ofthe particular mobile device and regions of interests of the user of theparticular mobile device.
 3. The method of claim 1, wherein the datastructures further include a list of one or more tracking events, videoevents, social events, area events, web request events, partner events,trackables, cameras, social threads, and areas, and wherein each listincludes identifications that reference said each list.
 4. The method ofclaim 1, wherein said prediction of what the user will do while on siteincludes one or more of predicting a future travel path of theparticular mobile device, predicting what web sites the user will visit,and what products or services the user is likely to purchase.
 5. Themethod of claim 1, further comprising utilizing said analyzed behaviorof the user and said prediction of what the user will do while on sitefor one or more of providing products, services and offers, productplacement and management, staffing support and scheduling, security orlaw enforcement services, and object or person tracking.
 6. The methodof claim 1, wherein the tracking information includes one or more ofduration of the user stay on the site, physical and web locationsvisited by the user, apps used by the user, user profile, productslooked at, and web browsing actions.
 7. The method of claim 1, whereinthe tracking information is obtained from one or more of a point ofsales terminal, an iBeacon device and a WiFi access point.
 8. The methodof claim 1, further comprising generating a heat map to show real timetracking, visitor density location popularity and flow analysis ofvisitors within the site.
 9. The method of claim 1, further comprisingutilizing said analyzed behavior of the user and said prediction of whatthe user will do while on site to transmit or display messages from oron an interactive terminal or digital signage screen to the user. 10.The method of claim 1, further comprising identifying previous paths ofthe user, and matching the previous paths with the video streams of themovement of the particular mobile device for further identification ofthe user.
 11. The method of claim 1, further comprising generating alertinformation when the user returns to the site.
 12. A non-transitorycomputer storage medium including a plurality of instructions, theinstructions when executed by one or more processor performing a methodfor tracking a mobile device in a site, the method comprising:receiving, in real time, a plurality of different identifications for aparticular mobile device visiting the site, wherein the plurality ofdifferent identifications include false or repetitive identificationsfor the particular mobile device; filtering out the plurality ofdifferent identifications for the particular mobile device to obtain atrue identification for the particular mobile device, wherein the trueidentification does not include false or repetitive identifications forthe particular mobile device; in real time, identifying cameras relevantto movement of the particular mobile device responsive to the trueidentifications for the particular mobile device; receiving videostreams of the movement of the particular mobile device from theidentified cameras, time stamping the received video streams and storingthe time stamped video streams in a computer storage medium; generatingdata structures for the video streams and tracking information of theparticular mobile device, the data structure including time stampedvideos, and viewpoints of the identified cameras; utilizing the datastructures to retrieve, in real time, video and tracking information forthe particular mobile device and the user, as the particular mobiledevice moves in the site; and applying analytics to the retrieved videoand tracking information to analyze behavior of the user and to predictwhat the user will do while on site.
 13. The non-transitory computerstorage medium of claim 12, wherein the data structures further includesone or more of a user of the particular mobile device, regions ofinterests of the user of the particular mobile device, a list of one ormore tracking events, video events, social events, area events, webrequest events, partner events, trackables, cameras, social threads, andareas, and wherein each list includes identifications that referencesaid each list.
 14. The non-transitory computer storage medium of claim12, wherein said prediction of what the user will do while on siteincludes one or more of predicting a future travel path of theparticular mobile device, predicting what web sites the user will visit,and what products or services the user is likely to purchase.
 15. Thenon-transitory computer storage medium of claim 1, further comprisinginstructions to execute identifying previous paths of the user, matchingthe previous paths with the video streams of the movement of theparticular mobile device for further identification of the user, andgenerating alert information when the user returns to the site.
 16. Asystem for tracking a mobile device in a site comprising: a plurality ofcameras coupled to a computer network; a wireless access point forproviding access to the computer network; a computer storage medium forstoring information; and a server coupled to the computer network for:receiving, in real time, a plurality of different identifications for aparticular mobile device visiting the site, from the wireless accesspoint via the computer network, wherein the plurality of differentidentifications include false or repetitive identifications for theparticular mobile device; filtering out the plurality of differentidentifications for the particular mobile device to obtain a trueidentification for the particular mobile device, wherein the trueidentification does not include false or repetitive identifications forthe particular mobile device; in real time, identifying relevant camerasfrom the plurality of camera relevant to movement of the particularmobile device responsive to the true identifications for the particularmobile device; receiving video streams of the movement of the particularmobile device from the identified relevant cameras, time stamping thereceived video streams and storing the time stamped video streams in thecomputer storage medium; generating data structures for the videostreams and tracking information of the particular mobile device, thedata structure including time stamped videos, and viewpoints of theidentified cameras; utilizing the data structures to retrieve, in realtime, video and tracking information for the particular mobile deviceand the user, as the particular mobile device moves in the site; andapplying analytics to the retrieved video and tracking information toanalyze behavior of the user and to predict what the user will do whileon site.
 17. The system of claim 16, wherein the data structures furtherinclude a user of the particular mobile device, regions of interests ofthe user of the particular mobile device, a list of one or more trackingevents, video events, social events, area events, web request events,partner events, trackables, cameras, social threads, and areas, andwherein each list includes identifications that reference said eachlist.
 18. The system of claim 16, wherein said prediction of what theuser will do while on site includes one or more of predicting a futuretravel path of the particular mobile device, predicting what web sitesthe user will visit, and what products or services the user is likely topurchase.
 19. The system of claim 16, wherein the server furtherutilizes said analyzed behavior of the user and said prediction of whatthe user will do while on site for one or more of providing products,services and offers, product placement and management, staffing supportand scheduling, security or law enforcement services, and object orperson tracking.
 20. The system of claim 16, wherein the server furtheridentifies previous paths of the user, matches the previous paths withthe video streams of the movement of the particular mobile device forfurther identification of the user, and generates alert information whenthe user returns to the site.